CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION Theory versus Reality in Charter Schools in Colombia

Charter schools are among the most prevalent public-private partnerships in the education sector. Too frequently, charter schools are assessed only by measuring outputs such as enrollment and test scores. This paper assesses the logic model behind charter schools, specifically the mechanisms of accountability and competition. This paper examines these mechanisms through study of the Concession Schools (Colegios en Concesión and hereafter referred to as CEC) in Bogotá, Colombia, using a realist evaluation methodology. The authors find that both accountability and competition do not work as conceived by charter school proponents. The findings of this paper demonstrate that successfully operationalizing the charter model represented by the CEC program requires that the mechanisms underlying each segment in the model are carefully designed, that they directly connect, and that the functioning (or not) of one does not adversely impact the others—a difficult task given the dynamic and sensitive nature of many of the mechanisms. More specifically, both a lack of competition in the bidding process and poorly designed contracts affect the ability of the government to hold schools accountable; similarly, insufficient choice for parents—or, parental choice on an insufficient scale—affects the development of competitive dynamics among schools. Keywords: charter schools, Colombia, realist evaluation, public-private partnerships, accountability, competition CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION Accountability and Competition for Charter Schools? Theory versus Reality in Concession Schools in Bogotá, Colombia At the broadest level, public-private partnerships (PPPs) can be defined as “cooperation of some sort of durability between public and private actors, in which they jointly develop products and services and share risks, costs and resources which are connected with these products” (Hodge & Greve, 2010, p.4). As should be clear from this definition, the PPP label can apply to multiple reforms, which promote private sector involvement (both individual and institutional) and introduce market-based values and mechanisms (Wolff, Navarro & González, 2005). In education, the most well-known PPP policies are vouchers and charter schools, though a wide range of other PPP arrangements exist. In education, not only have PPP educational policies gained traction globally in the past 10 years (ADB, 2010; Patrinos, Barrera-Osorio, & Guáqueta, 2009; Robertson, Mundy, Verger, & Meashy, 2012; UNICEF, 2011), but they appear to be increasing in influence. There are many indications of this, such as the continuing growth and rising political clout of privately managed schools in the United States, the inclusion of PPPs as a central theme set forth by the European Union in its EU2020 vision for sustainable and inclusive growth, and the prominence of PPPs in the process of defining the post-2015 development agenda for education (Au & Ferrare, 2014; European Commission, 2010; Fabricant & Fine, 2012; United Nations, 2013). In promoting PPPs, policy entrepreneurs often highlight four pathways through which these arrangements can lead to positive educational outcomes. These include: (a) operational flexibility of private entities (i.e., the ability to customize one’s organization and service 1 We are grateful to Toni Verger for his assistance in the framing of this introduction. 2 These other arrangements include any instance in which private actors contribute some aspect of the provision of public education, ranging from building construction to cleaning services, for example (Patrinos, Barrera-Osorio, & Guáqueta, 2009). CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 4 offerings), (b) bidding and selection processes (through which government agencies select private companies to provide a given service), (c) risk-sharing between the government and private sector (such that private companies will enter the market, knowing that they do not bear the full burden of failure), and (d) competition (through which public schools react to the threat of losing students) (Patrinos, Barrera-Osorio, & Guáqueta, 2009). However, despite the somewhat specific nature of these pathways, and despite the many assumptions embedded in the operation of each one, the debate in education tends to boil down, more often than not, to findings related to such outcomes as student achievement or student dropout rates. (Fabricant & Fine, 2012; Lubienski, 2003; Lubienski & Lubienski, 2006; Miron & Nelson, 2002). The problem with this is that essential aspects of the operation of PPPs are overlooked. If debates and studies tend to be restricted to input-output research that assesses outcomes alone without unpacking the mechanisms or pathways by which those outcomes are achieved, then essential information is missing. On one hand, we are left with a partial understanding of the various interactions and processes that connect private providers and government agencies; on the other hand, we are left with an incomplete view of the dynamics among schools in competitive environments. The present paper attempts to address this issue through the presentation of findings from a study of charter schools in Bogotá, Colombia. The charter model of interest is known as Concession Schools (or, in Spanish, Colegios en Concesión, CECs). It began in 1999, is intended to provide education that is high quality, privately managed, and publicly funded to poor students in specific areas of Bogotá. This program has become a well-known and relatively high-profile reform. It has been promoted by the World Bank (see, e.g., Patrinos, Barrera-Osorio, & 3 It is worth noting, as well, that broad reviews have shown the impacts of charter schools on student achievement to be mixed at best and typically no better than traditional public schools (CREDO, 2009; Jeynes, 2012; Lubienski & Lubienski, 2006; Zimmer, Gill, Booker, Lavertu & Witte, 2012). CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 5 Guáqueta, 2009) and its founder has continued to feature in key conferences in the United States on private education led by such actors as the International Finance Corporation (2014) and the Brookings Institute (2015). This program thus reflects education reform trends in the United States and internationally while also serving as a reform example itself that encourages the continuation of the PPP trend. These characteristics make the CEC program particularly salient for understanding multiple aspects of how charter school programs operate. In the present paper, we will trace and unpack—through the application of a realist evaluation methodology—each of the mechanisms embedded in the theory of action of the CEC program. As will be shown, the findings from this study have implications for the extent to which mechanisms of accountability and competition can be expected to materialize in practice through the introduction of charter-type programs. Importantly, the relationships of interest are not only those between parents and schools but also those between charter management organizations and the central office managers who oversee them. Notably, these relationships are an under-studied aspect of charter school programs, as the literature review in the next section makes clear. To address the above-mentioned issues, this paper moves through a number of sections. First, we review the relevant theory and evidence on charter schools. Second, we detail the methodology employed for the study. Third, we situate the CEC program within Colombia by explaining the national and city-level context out of which it emerged. Fourth, we then discuss the specific policy provisions and the theory of action that guide this program. Fifth, we present the results of this study, which relate specifically to the processes of accountability and to the dynamics of competition that accompanied the Concession Schools. The final sections offer discussion, implications, and conclusion. CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 6 Theory vs. Reality in the Literature The literature on PPPs generally and on charter schools specifically has discussed for many years the theory of accountability and competition in education markets (Chubb & Moe, 1990; Lubienski, 2006; Patrinos, Barrera-Osorio, & Guáqueta, 2009). The former typically relates to the ability of a school district, for example, to monitor and, if necessary, to take action against those private entities that are providing services related public education (Crew & Anderson, 2003). Key aspects of this arrangement are the use of contracts and open-bidding processes. In theory, the open-bidding process should be transparent and competitive and the contracts themselves should not only be awarded on the basis of past performance but should also include clear stipulations regarding performance requirements (Patrinos, Barrera-Osorio, & Guáqueta, 2009). Subsequently, those officials responsible for overseeing charter schools or other service providers can carefully monitor administrative behavior and/or academic outcomes, thereafter using poor performance as a justification for revoking charters or for cancelling contracts (Crew & Anderson, 2003). Of course, the logic of this arrangement presumes that performance criteria are clearly established, that public managers regularly collect the necessary decision-making information, and that those public managers have the ability to exercise their authority (Lubienski, 2006). The idea of competition in education markets also rests on a number of assumptions. In the case of charter schools, the primary assumption is that, as a result of the creation of multiple choices from which parents can choose, schools will be pressured to improve the quality of the education offered (Chubb & Moe, 1990; Lubienski, 2003). Improvement is thought to be achieved through administrative and pedagogical innovation (Lubienski, 2003) and/or simply by strategically employing the school’s resources in order to improve indicators such as test scores CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 7 or dropout rates. Importantly, though, in order for competition to work in this way—that is, in order for an education market to function properly—not only must eligible parents be aware of and able to exercise their choices but school principals (or the umbrella company for the school) must also be incentivized to respond by the perception that other schools represent a legitimate threat to their survival. To the extent that these dynamics are realized, theory suggests that service providers will attempt to minimize their costs and will use their resources more efficiently, thereby leading to an education system that is more efficient overall (Rho, 2013). The available evidence, however, suggests that accountability and competition often do not function in practice as they are presented in theory. For example, on the issue of accountability, Crew and Anderson (2003), in their study of 103 charter schools in Florida, found that state law required charter schools to produce annual reports that contained information on school finances, faculty and staff, and the attainment of performance goals. However, in practice, the reports submitted by the schools complied with few of these legal requirements in that they “stated objectives in ways that could not be tracked or measured, provided very little baseline data and often made few references to the progress they had made toward the goals identified in their charter applications” (Crew & Anderson, 2003, p. 200). Moreover, “no school complied with the requirement that baseline data be presented that would permit a showing of the academic progress made by individual students during the time enrolled in the charter school” (p. 200). Compounding this situation was the fact that “no one in Florida’s educational system [took] responsibility for annual analysis and reporting of data” (p. 200) and that “the Charter School Office claimed to have no mechanism to enforce [the annual report] requirement on recalcitrant schools” (p. 200). Consequently, not only is there little information about the CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 8 performance of charter schools but these schools have been able to operate with performance and “without repercussion” (p. 202). Other studies have shown similar breakdowns in relationships of accountability. Acar and Robinson (2004) conducted a study of accountability in the United States between K-12 public schools and private or non-profit organizations. On the basis of 38 interviews across 17 states, they found that four issues primarily affect accountability: (a) insufficient personnel on the part of the schools to monitor the private and non-profit partner organizations, (b) insufficient monetary resources to collect the necessary information, (c) difficulty in measuring service outcomes, and (d) inability to access the necessary information regularly and in a timely fashion. Other issues found to hinder the exercise of accountability were personal differences, communication issues, and frequent turnover in key personnel (Acar & Robinson, 2004). Thus, as one might expect, even when service provision arrangements are governed by contracts, there can be significant transaction costs that stem from untrusting and unstable relationships and a lack of information (Reeves, 2008). Separately, after interviewing 41 public procurement specialists, Diggs and Roman (2012) found that accountability is not always a straightforward or static concept. As pointed out by those whose job it is to hold private providers accountable, the way in which this concept is defined varies based on an individual’s experience, organizational position, and career aspirations. Accountability can be defined as meeting legal requirements but it can also be interpreted as meeting informal or unofficial expectations, for example (Diggs & Roman, 2012). In terms of bidding processes, very little research has been conducted. Though it is assumed that multiple private or non-profit organizations will compete for public sector contracts, what happens when this is not the case? Do public administrators award contracts even where CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 9 there is insufficient competition and, if so, with what implications for corruption and the ability to effectively manage and, where necessary, to punish these schools when they know that their governmental supervisors cannot easily replace them (if at all)? While these questions are in need of answers—especially in reference to charter schools—one study examined a bidding process wherein private companies were sought as partners for school construction in Portugal (Da Cruz & Marques, 2012). Here, not only did the Portuguese government decide to work with the only willing private sector partner, but, in addition, it was found that this company could not “serve two masters” in that it continued to prioritize profit making over public sector needs (Da Cruz & Marques, 2012, p. 756). Research on competition similarly problematizes notions of a smoothly functioning market mechanism (Lubienski, 2003). For example, as pertains to information access, Alegre and Benito (2012) reveal that parents in advantaged socio-economic positions are better able to access privileged information (i.e., insider information on a school through social networks) in comparison with marginalized families, who must rely on “under-capitalized channels” (i.e., publically-available information). This kind of information asymmetry influences the choice sets that parents construct when exploring options for their children (Waslander, Pater & van der Weide, 2010). It can also lead to greater segregation of students along socio-economic or racial characteristics (Ball, Bowe & Gewirtz, 1995; Reay, 2004). The other side of the informational coin is that schools themselves can regulate what is known about them, and by whom. Lubienski (2007) concludes that school processes are opaque to outsiders, that schools enjoy “informational advantages over consumers” and that they have “disincentives to empower consumers with better, more actionable information” (p. 123). On this point, studies have shown that charter schools and private providers share only select information, CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 10 or choose not to make information about their school widely available, precisely in an effort to influence the composition of their student body (Lubienski, 2003, 2006; Jabbar, 2014). Another strategy is to focus on how the school is perceived—that is, to focus on marketing materials and the use of symbolism to evoke emotion and to create positive mental images (Jabbar, 2014; Johnsson & Lindgren, 2010; Lubienski, 2003). Interestingly, research has found that marketing is also a strategy to which principals turn when they sense competitive pressure for their students, as opposed to focusing on improving educational quality (Kasman & Loeb, 2013; Loeb, Valant & Kasman, 2011). These same studies further investigated the circumstances in which principals sense competitive pressure. While geography—or the distance between traditional public schools (TPSs) and a potential competitor—strongly influences which schools principals perceive as competitors, researchers also found that principals tend sense competitive pressure from those schools that are similar in terms of student achievement, race, poverty status, and enrollment size (Kasman & Loeb, 2013; Loeb, Valant & Kasman, 2011). Yet other studies have suggested that the perception of competition depends on the purpose of charter schools, for example, and the context into which they are introduced. Not surprisingly, TPSs may not perceive charter schools as a threat if the latter are intended to serve marginalized or low-performing students and if the TPSs experience no negative financial impact as a result of the charter schools (Waslander, Pater & van der Weide, 2010). A penultimate point to be made about competition is that it may not lead to pedagogical innovation. In a review of 56 studies related to innovation in charter schools, Lubienski (2003) found that administrative innovations are possible and common due to the autonomy that charter schools enjoy, but that there is no trend of classroom-level innovations; rather, “classroom CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 11 practices are relatively familiar and often revert to traditionalist modes of curriculum and instructions” (p. 396). To the extent that classroom practices are innovative, this is only the case in reference to local contexts; that is, a charter may adopt a pedagogical practice that is uncommon in its immediate geographic location, but charter schools do not fulfill the “R&D assumption of market-based school reform (p. 416). Lastly, there is surprisingly little evidence on the link between competition and efficiency, though proponents often highlight this connection (Chubb & Moe, 1990). What evidence does exist suggests that the introduction of market mechanisms does not induce cost savings. Levin (1998), for example, concluded that the costs associated with overseeing a system of vouchers would actually increase system-wide administrative costs past their current levels (and without a concomitant rise in test scores). More recently, Rho (2013) analyzed the impact of contracting generally (e.g., for test development or cafeteria management) in more than 1,000 school districts in Texas over 12 years. Though her results do not speak directly to the issue of efficiency or total cost, she concludes that “More contracting generates greater bureaucracy, more bureaucrats presage more contracting, and the cycle continues,” given the demands on school district administrators to supervise contractors through new managerial systems (p. 335). Going forward, however, additional studies are clearly needed on the issues of cost-effectiveness and efficiency, especially given their prominence in debates around PPPs generally and charter schools specifically. Just as additional research is needed generally on the mechanisms of accountability and competition, so too is more research needed on the Concession Schools of Bogotá. It is the case that a couple quantitative studies have been produced on the effects of CECs on student achievement (Barrera-Osorio, 2006; Bonilla, 2011), but, as pointed out by Edwards (2014), the CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 12 findings of these studies are open to interpretation due to issues of endogeneity and the lack of a true counterfactual. Given these issues, the present paper takes as its focus the inner-functioning of the program’s presumed accountability and competition mechanisms, as explained further in the next section. Methodology Our approach is based in systems thinking (Meadows, 2008). This means, first, that we see each element in the program’s theory of action as connected, and that changes in each are assumed to move through the system in a dynamic manner. Second, it means that we see interorganizational relationships, evaluation, information sharing, and communication as integral to maintaining and operationalizing each link in the casual chain. Methodologically, in order to operationalize this approach, we drew on realist evaluation (Pawson & Tilly, 1997; Pawson, 2006). This type of evaluation is ideal for the present study in that this study has been developed to help unpack the inner working of a program. That is, rather than asking what the effect of a certain program is, the idea is to trace and investigate the logic of that program. The benefit of this approach is that one produces findings that are specific to the set of circumstances in which the program is embedded. Put differently, producing such findings is advantageous because it helps to construct a body of knowledge about the outcomes that can be expected from different programs—with their specific logics—given particular technical, institutional, and political characteristics of the context. This approach is additionally appropriate for the study of the CEC program because, as noted in the previous section, prior studies have not inquired as to how the program works, but rather have focused on whether or not presumed mechanisms have produced certain outcomes. Thus, by presenting the results of a realist evaluation, we seek (a) to clarify the details of the CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 13 program’s logic model, (b) to analyze the extent to which (and whether or not) the assumed theory of action operates in practice, and (c) to derive lessons for the reform of such programs. Data Collection Data collection occurred during May-December 2013. For data collection, a number of specific strategies were employed that aligned with the purpose of this study and with the realist evaluation approach. Specifically, data collection entailed: gathering program evaluations (both academic and “gray” in nature), seeking out relevant literature (e.g., related to the historical, political, and institutional context of education in Colombia), conducting interviews and focus groups with a range of key actors who work within and outside the country’s education system, visiting CECs and TPSs, collecting documents from individual schools, and assembling statistical information on the schools in the final sample (more on this below). Importantly, these strategies allowed us not only to identify the logic model for the CEC program but also to compare the theory of the program with its reality in practice. A total of 34 persons were interviewed. The individuals interviewed were specifically selected because they occupied key positions in the education system in Bogotá. Because of their positions, they had intimate knowledge of the CEC program that pertained, for example, to program’s origins, the relationship between CECs and nearby TPSs, the dynamics of interaction between charter management organizations (CMOs) and the relevant managers within the Bogotá School System (Secretaria de Educación Publica, or SEP), the CMO bidding process, and the administrative strategies and behavior of the CECs themselves, in addition to other aspects. As can be seen from Table 1, interviews were conducted with officials from the Ministry of Education (MINED), representatives of the SEP in Bogotá (including those whose job it was to evaluate the CECs), directors of CMOs, principals from both CECs and their neighboring CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 14 TPSs, and parents, among others. To give an idea of the valuable and internal perspective gained from participants, consider that in-depth interviews were conducted with the Minister and a Vice-Minister of Education for Colombia during 2002-2010, two Secretaries of Education for Bogotá during 1998-2003, the Director (as of 2013) of the Colombian Institute for Educational Evaluation (ICFES, for its name in Spanish), the first Director of CEC expansion for the MINED, the Director of Educational Coverage for Bogotá during 1999-2002, and the Director (as of 2013) of Evaluation for the SEP of Bogotá, as well as others. [TABLE 1 HERE] To understand the administration of CECs, the principals of five separate Concession schools were interviewed from a single locality in the southern part of Bogotá. This locality was chosen for its high concentration of CECs (with 20 percent of them located here) as well as for its population density, since previous research has shown that competition mechanisms tend to function better in such areas (Elacqua, Schneider & Buckley, 2006). In addition to performing school visits and interviewing all CEC principals in this particular locality, the principal of the TPS nearby to each CEC was also sought out for interviews. In all, 10 principals were interviewed: five from CECs and five from TPSs. Finally, the Director of the SEP office in that locality was also interviewed, in order to grasp the extent to which competitive dynamics in fact developed. Data Analysis Analysis entailed a number of stages. The first stage required that we interrogate the documents and archives collected in order identify the theory of action of the program. This was particularly essential not only because this theory of action guided subsequent analysis, but also 4 For three of the five CECs in the locality of focus, one nearby TPS principal was interviewed; for one of the CECs, two nearby principals were interviewed; and for one CEC, it was not possible to interview the nearby TPS principal. CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 15 because proponents of this program have tended to emphasize mechanisms that were not initially intended to be primary drivers of school improvement. The theory of action for the CEC program is presented in a later section of this manuscript. The second stage then focused on understanding how each facet of the CEC program operates in practice, in addition to the pressures and constraints that impact the program’s functioning. As part of understanding these pressures and constraints, we paid particular attention to the way in which—and the context in which—the program initially developed, in order to be aware of the institutional dynamics that could subsequently impinge upon the program. In the third and final stage, and in accordance with realist evaluation, we then systematically traced in practice each link in the causal chain that is theorized to produce the program’s outcomes. The results of this analysis are the focus of the present paper. Before presenting those results, however, we first detail the program’s antecedents. Context National Context and Policy Trends In the 1980s, Colombia faced a variety of challenges, chief among them security and political inclusion. The decade was marked by an increase in drug trafficking, and protracted conflict between guerillas, paramilitary forces and the national government. Many Colombians felt excluded from their political system, a two-party system long controlled by those from privileged backgrounds (Palacios, 2007). During this time the Colombian government began implementing neoliberal economic reforms, in part due to pressures and recommendations from the World Bank and International Monetary Fund (Estrada Alvarez, 2005). However, this ideological shift cannot solely be attributed to external pressure; many Colombian economists were educated in the United States in conservative economics departments. Additionally, the CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 16 Ford Foundation and University of Chicago established think tanks in Colombia, which promoted neoliberal thought (Estrada Alvarez, 2005). Both external actors and Colombian technocrats also worked through the National Planning Department, which was politically influential, ideologically conservative, and technically savvy. Importantly, this agency is considered to have filled a void left by the lack of capacity of the other governmental ministries, particularly with regard to social and economic planning. Politically, the late 1980s and early 1990s were also a time of increased political inclusion through both a student movement and the demobilization of some guerilla groups. The latter would be included in the political process through the formation of official political parties. In 1991, a new, liberal constitution was adopted by the National Assembly, as it was now comprised of “practically all social sectors, including some recently demobilized guerrilla groups, indigenous movements, and unions, especially the Colombian Federation of Education” (Miñana, 2010b, p. 34). The 1991 constitution guaranteed public education for the first time, mandating education from five years of age to 14 (Miñana, 2010a; MINED, 2010). The new constitution also called for additional funding for education, improved educational outcomes and a decentralization of management. In 1993, legislation required that a minimum amount of funding, the ‘situado fiscal,’ be transferred by the MINED to local authorities to fund health and education. In 1994, municipalities with sufficient capacity were then allowed to directly manage their human resources and the situado fiscal (Meade & Gershberg, 2008). 5 By 1998, all 32 departments and 4 independent school districts had been certified and were administering basic education (MINED, 2010). The stipulation that departments must have 100,000 inhabitants in order to have administrative tasks decentralized was not introduced until 2002. CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 17 While the situado fiscal was intended (in theory) to be proportionate to the number of beneficiaries (i.e., students) and to account for the administrative needs of the municipality, in practice this led to costs for the MINED that increased drastically and were distributed inequitably (MINED, 2010). This is because funding was based on administrative costs and the number of teachers (not students) located in each departmental government. Departments with more (and more highly qualified) teachers received larger disbursements than those departments that were more difficult to staff or staffed with less-qualified teachers. Some departments went so far as to hire additional teachers or to misclassify expenditures to receive additional funding from the MINED (MINED, 2010). The situado fiscal formula did not consider student enrollment or student achievement. There was thus little incentive for departments to focus on student improvement or to better teacher coverage. By the late 1990s, with 20 percent of the national budget allocated to education, teacher salaries made up 90 percent of national education expenditures, while only 1 percent of spending was dedicated to materials (Villa & Duarte, 2005). In contrast, only 12 percent of local revenues from departments, districts, and municipalities were dedicated to funding education (Villa & Duarte, 2005). With the onset of a severe economic crisis in the country in 1998, increased resource consumption by the departments increasingly conflicted with budgetary constraints faced by the MINED. Key policy actors were thus looking for ways to reduce spending on education and to make it more efficient and effective. 6 In 2001, new legislation sought to address these problems by centralizing human resource policies, teacher evaluations and assignments. Larger departments and cities would maintain some responsibility for hiring and firing, but funding would now be based on student population served including: teacher type, student to teacher ratio, socio economic status and the percentage of rural population. Funding in excess of required expenses, including teacher salaries, could be spent on other eligible expenses including infrastructure. This legislation forbade teacher pay increases in excess of available funds, and allowed for the private contracting for service provision, including building maintenance (Meade & Gershberg, 2008). CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 18 Context of Bogotá In 1995, Bogotá assumed management of its education system and encountered problems similar to those described above. The number of teachers employed tripled in 1996 to 33,172, yet student-teacher ratios varied widely due to a lack of information and poor management practices. It was difficult to assign teachers and financial resources based on educational needs and, as such, student-teacher ratios varied widely across schools. Enrollment projections were unreliable and educational demand across the city was poorly understood. As of 1997, the net enrolment rate at the primary level for Bogotá was 90.7 percent, while at the secondary level it was 83 percent (Castro, Pérez, & Alvarez, 2012). In 1998, following the election of Enrique Peñalosa as mayor, Cecilia Vélez was appointed the Secretary of Education for Bogotá. During the early 1990s, when decentralization began, Veléz had worked at the National Planning Department. After obtaining a master’s degree in urban and regional planning from the Massachusetts Institute of Technology, Veléz would serve as Bogotá ́s Secretary of Education from 1998 to 2001, and would later become Minister of Education for Colombia under President Uribe. During her tenure as Secretary of Education for Bogotá, Vélez sought to address some of the systemic problems that she perceived. Vélez had three goals: enhance student results, increase access and retention, and improve management (Castro, Pérez & Alvarez, 2012). Vélez instituted a number of reforms including student-teacher ratio parameters and standardized testing at select grade levels. However, the most notable reform of her tenure was the introduction of Concession schools. This initiative addressed the availability of schools in marginalized locations with high demand. Through a bidding process, successful private schools CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 19 would construct and manage schools with public funds, as described in more detail below. A total of 25 Concession schools were constructed between 1999 and 2003. CEC Policy Emergence of CEC Policy Although Colombia had experimented with a voucher program (known as PACES, Plan de Ampliación de Cobertura de la Educación) in the 1990s, it acquired a negative reputation—at least among some actors in the education sector—because lower quality private schools emerged to take advantage of the program. It was for this reason that Cécilia Vélez, with her lack of faith in the public school system, searched for alternative arrangements to contract private providers in 1998 when she entered office as the Secretary of Bogotá’s SED. Notably, her belief in the fast reaction time of private schools combined well with the desire of Mayor Peñalosa to act quickly to expand access and quality to the poorest students in the city. After considering a number of options (e.g., putting private school principals in public schools or creating school boards to oversee public schools), a public-private model won out. Successful private schools would manage newly constructed public schools in targeted areas of high need. This idea would become the Concession Schools program. Somewhat surprisingly, the teachers unions did not oppose the implementation of the CEC program and its implied privatization of public resources. Villa and Duarte (2005) note that this was because the CEC program did not affect teachers working under their existing bargaining agreement, and because teachers hired to work at CECs would receive comparable salaries, though without tenure or other additional benefits. Policy Provisions 7 See Miñana (2010b, p. 174) for more on external influence on Colombian education. CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 20 Villa and Duarte (2005) and Bonilla (2011) have most clearly laid out the legal, institutional, and theoretical facets of the how the CEC program was supposed to work. The legal and institutional aspects of this program are: • Contract duration: Initial CEC contracts were established for 15 years—1999-2014. Future contracts were also written to terminate during 2014. The shortest contracts granted were for 11 years, for those schools which opened in 2003. • Selection of CMOs: Eligible CMOs are organizations that already operate schools (typically private schools) which agree to operate an additional school constructed and funded with public money. Participating CMOs are to be selected—through a competitive bidding process—on the basis of their educational quality, as gauged by their performance and reputation. • CMO obligations: CMOs have to provide formal education services in the form of a full, single-shift school day at the primary and secondary school levels to children from poor backgrounds, on specific premises, in return for a per capita fee. Half-day shifts are provided to preschool students. CMOs also have to provide each student with one meal daily, have to meet minimum governmental curriculum requirements, and must have the same academic year duration as TPSs, among other requirements (Bonilla, 2011). • CEC school infrastructure, features, and cost: CECs were newly constructed specifically for this program, in marginalized areas with insufficient capacity to serve the student population. Beyond simply having classrooms, CECs are intentionally well-resourced, containing such features as recreation rooms, science laboratories, art rooms, technology rooms, a library, and multi-purpose fields. The average cost of each school is $2.5 8 Note that this statement does not apply to the renewal of contracts in 2014 for certain schools. CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 21 million, including the purchase of the land, construction, and school furniture and equipment (Villa & Duarte, 2005). According to Villa and Duarte (2005), not only should the quality of the building and its facilities be a source of pride to the children and the community, and not only should these schools raise expectations about the learning experience, but they should also be centers of social development for their communities. • School and teacher management requirements: CMOs enjoy autonomy in school management, especially in the hiring of teachers and principals, though teachers must meet the legal requirements for their profession. Additionally, while teachers are paid in accordance with the national pay scale for public school teachers, they do not benefit from the job security conferred by the national Teaching Statute to teachers in TPSs (Villa & Duarte, 2005). As such, CECs are able to hire teachers on ten-month renewable contracts from a nonunionized pool of applicants, though some CECs offer longer contracts, up to 11.5 or 12 months. The implication is that—in contrast to TPSs—CECs have the ability to adjust their teaching corps each year (Bonilla, 2011). • CEC finance: At the outset of the CEC program, the government funded the CECs at a rate of $1,200,000 (USD$520) Colombian pesos per student per year, in comparison with about $1,000,000 pesos (USD$430) for TPSs (Barrera-Osorio, 2006). More recently, in 2013, CEC funding ranged from a low of $1,325,379 pesos (USD$689) to a high of $1,916,411 pesos (USD$996) for the best academically performing CEC, according to the evaluation of the Institute for Educational Research and Pedagogic Development’s 9 In practice, though CMOs would like to design incentive schemes, this is difficult. According to Villa and Duarte (2005), “The need to keep real costs flat limits its ability to offer a career path to all teachers, but [the CMO] also has to guarantee an education process that delivers the levels of achievement agreed to in the contract. Thus it must retain and foster a core of highly motivated and capable teachers to set academic standards for another smaller or larger group, whose members must regularly be changed to prevent staff costs from becoming too high” (p. 119). CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 22 (IDEP, in Spanish) (2010). It is not clear why the per student funding amount for CECs has diverged so significantly. 11 It may be the case that, on average, TPSs continue to receive less on a per student basis, since CECs are awarded annual per student funding increases in line with inflation. After expenses, any surplus or loss is the responsibility of the concession-holder. • Student admission/selection: Clear eligibility and admission criteria are set for CEC recipients: students must belong to the lowest socioeconomic strata, and live near the school (Villa & Duarte, 2005, p. 114). The schools themselves are responsible for carrying out the selection process. Schools cannot use admission tests to make their decisions, and they must admit the most economically disadvantaged students first. Students for whom there is not space in the CECs are assigned to the nearest school to their residence with an available seat (Bonilla, 2011). • Accountability between government and CMOs: According to the contract between the SEP and the CMOs, there are three accountability dimensions. Through periodic evaluations by the SEP, CMOs are accountable, first, for building management, second, for proper staff management, and, third, for student learning. For this last dimension, the CMOs not only have to submit to an independent assessment but also have to be rated “good” or “very good” every year; test scores may not be “unsatisfactory” for two 10 See document “Valor Niño Concesión, 1999-2013” for more on per student funding for all CECs since the program’s inception. Currency conversions listed here used exchange rates as of January 10, 2014. 11 For reasons that are unclear, Villa and Duarte (2005) report different findings on CEC finance. According to them, CMOs received $475/per pupil per year at the outset, which was less than per pupil unit cost of $595 unit cost that they reported for Bogotá’s TPSs. 12 One TPS principal reported that he receives about $1,000,000 pesos per student, which, if accurate, reflects the same amount received by TPSs 15 years ago (BOGACT15). 13 Students’ families must have income equivalent to level one or two of the “SISBEN” system (the System for the Selection of Beneficiaries of Social Programs), which is a “six-level classification used by the local government to characterize the socio-economic characteristics of a given dwelling. ... In general, residents of dwellings in the first 2 strata are considered the urban poor and pay, for example, subsidized prices for public utilities such as water and electricity” (Bonilla, 2011, pp. 16-17). CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 23 consecutive years. In the event that test scores fall below this threshold, the contract stipulates that the charter may be terminated (Villa & Duarte, 2005). Theory of Action The implicit theory of action—or program logic—can be derived from the preceding sections. Based on the understanding we have of (a) why the program was developed by its protagonists and (b) how the program is supposed to operate in terms of its legal, institutional, and theoretical dimensions, we can visually depict the way that the program’s various elements and mechanisms should ideally function in relation to one another—and with what outcomes. This visual representation can be found in Figure 1. In line with the theory of charter schools more generally, proponents of the CEC program suggest that certain dynamics and outcomes should result from the above-mentioned policy provisions. First, it is thought that the schools in this program should be more efficient because they are selected through a competitive bidding process, and because the CMOs face lower labor costs. Second, it is thought the CECs should respond to the threat of accountability from the SEP, in that they can be closed if they do not perform well for multiple years on student achievement tests. Third, and finally, they suggest that nearby TPSs should feel a sense of competition with the CECs, because they may lose students if they do not improve the quality of education offered (Barrera-Osorio, 2006; Bonilla, 2011; Villa & Duarte, 2005). The logic model provided below for the CEC program is integral to the realist evaluation presented in this paper. This is because the logic model guided both the collection and analysis of data. By collecting data on each of the logic model’s dimensions and then analyzing that data against how the CEC program is supposed to function, we can identify the strengths and weaknesses of the program’s approach. CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 24 Before proceeding, it should be noted that in the logic model the mechanism of parental choice is a different color than the rest of the diagram. This is done to reflect the fact that this mechanism was thought to be a secondary mechanism for school improvement. Together, the bidding process, the school evaluation system (based on student achievement), and the threat of government accountability were thought to be the primary drivers that would lead to superior performance of the CECs. [FIGURE 1 HERE] Profile of CECs in Bogotá During 2005-2011, the CECs enrolled between 3.7 and 4 percent of students in Bogotá. Of the 39,667 students in CECs as of 2012, 7 percent were in preschool, 79 percent were in grades 1-9, and 13 percent were in grades 10-11 (Secretaría de Educación, 2013). As for the schools themselves, they are located in the marginalized, peripheral areas of the city, with the majority (80 percent) located in the western and southern parts (Bonilla, 2010). With regard to the CMOs, documents show that nine different organizations have granted charters. Three of these nine were “family benefit societies” (cajas de compensación, in Spanish). As Villa and Duarte (2005) explain, these are “private entities funded by 2 percent of the payroll of private companies and public institutions. Their purpose is to provide recreational, health, training, housing, and other services to affiliated workers” (p. 122). Importantly, those that have participated in the CEC program “have extensive experience in formal and non-formal education” (p. 122). The rest of the CMOs have been created—and are run—by members or affiliates of elite private schools, religious orders, or university-private school alliances. Findings CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 25 As indicated, our focus is the inner-workings of the CEC’s assumed mechanisms and the factors that influenced the functioning of those mechanisms. For an extended discussion of the available evidence on CEC program outcomes, see Edwards (2014). As noted there, access certainly did increase as a result of the construction of 25 CECs in marginalized areas with few schools, with these schools serving just under 40,000 students (or 4 percent of all students in Bogotá) in 2012. On the other hand, data on efficiency and quality were either unavailable (in the case of efficiency) or incomparable (in the case of educational quality). That said, we now turn our attention to contextual, institutional, technical and process dynamics around accountability and competition in the CEC program. As we discuss below, these issues combined in various ways to prevent the theory of action from being realized in practice. Following the flow of the logic model presented above, our discussion of findings is spread across multiple sections. The issue of accountability is divided across the following subsections: (a) bidding process, (b) evaluation system, and (c) governmental accountability actions. Subsequently, the issue of competition is divided among: (a) information system, (b) parental choice, and (c) school behavior. These grouping reflect the components that should combine in practice to produce properly functioning mechanisms of accountability and competition. Components of Accountability Bidding process. The findings on the bidding process are surprising; the initial process lacked competition. First, this was because the SED determined the entities they wanted to serve as CEC management organizations based on the test results of their existing schools. The SED then worked backwards to determine the criteria that they could require in the bidding process (NATACT7). CHARTER SCHOOL ACCOUNTABILITY AND COMPETITION 26 Second, the lack of competition resulted from the bidding guidelines. The point here is that applicants—from the pool identified by the SED—had to meet extensive criteria regarding the provision of services, and they had to do so for a level of per-pupil funding that was fixed (at the beginning). The implication is that is was difficult for organizations to make themselves eligible given the bidding requirements (NATACT6). One consequence of this situation is that the SED actually had to persuade a number organizations to participate in the bidding process (NATACT4). Third, the lack of competition was due in part to the initial skepticism of qualified bidders. The Jesuits, nuns, and leaders of elite private schools were not only confused as to the extact nature of the CEC program, but also had to be convinced that the government would provide continual funding for the program (NATACT6). The quote below, from the CEC program’s first coordinator wihtin the SED, adds further depth to these issues: It was my job, I went to the schools we selected, we made a list of schools, ... I met with the board of directors of all of the schools, of those we had chosen, and I presented the project, and it was a disaster, but I convinced them. For example, I remember meeting with the board of directors of the Gimnasio Moderno, some old men who didn’t understand anything, and they said, “this is going to cost us,” and I explained that it wasn’t going to cost them anything, and they asked where the school was going to be, and I said in Icará, and they said it was going to be very difficult, and it was like this with all of the groups, even more with the religious communities. I had to present to them and explain to them and a have thousand meetings, show them the contract ... With the nuns it was absolutely impossible, ... [they] didn’t understand anything (NATACT6, int 1, p.